Device identification device, device identification method, and device identification program

ABSTRACT

A device identification apparatus includes: a communication information collection unit configured to acquire communication information of existing devices and an identification target device; a feature amount generation unit configured to make the communication information of the existing devices&#39; feature amounts and assign labels to generate first training data, make the communication information of the identification target device feature amounts and assign a dummy label to generate second training data, and further acquire communication information of the identification target device to generate identification data; a machine learning unit configured to cause a learning engine to learn the training data, and input the identification data to classify the identification data into the labels; a degree-of-similarity calculation unit configured to calculate a degree of similarity for each label; and a device identification unit configured to use a new type determination threshold to determine whether the device is a new type device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Stage application under 35 U.S.C. § 371of International Application No. PCT/JP2019/033116, having anInternational Filing Date of Aug. 23, 2019. The disclosure of the priorapplication is considered part of the disclosure of this application,and is incorporated in its entirety into this application.

TECHNICAL FIELD

The present invention relates to a device identification apparatus, adevice identification method, and a device identification program thatidentify a device connected to a network.

BACKGROUND ART

Today, the Internet of Things (IoT) continues to expand rapidly, and awide variety and a large number of devices are being connected to anetwork. There is a forecast that 50 billion devices are connected tothe Internet in 2020, and more and more devices are expected to beinstalled in a variety of environments such as homes, factory, andstreets. Moreover, there are various types of devices connected to anetwork, including sensors such as a camera or a thermometer, smallcomputers such as a smartphone, actuators such as a speaker or adisplay, and the like, and computational processing capabilities andprotocols vary widely. An administrator of devices in each environmentis required to accurately grasp and manage a nature and a state of eachdevice so as to properly and safely use a wide variety and an enormousnumber of devices described above.

Examples of information that is influential in managing IoT devicesinclude a “class” and a “model” of a device. Here, the class of a devicerefers to a rough classification corresponding to a function such as acamera, a speaker, a printer, a smartphone, a personal computer, or thelike. The model is information capable of specifically identifying aproduct, which can be expressed as model number xx available from ACorporation, for example. For a device connected to a network, when anadministrator can grasp these pieces of information, it has a greatadvantage for asset management of devices or service utilization.Accordingly, it is required that only connection to the network canautomatically display these pieces of information for the administrator.

For a device connected to a network, there is disclosed a technique ofusing communication information to identify a device (see NPL 1). Atechnology described in NPL 1 extracts a feature amount fromcommunication information transmitted and received by a device in anetwork, and calculates a degree of similarity with accumulated datacollected by a similar procedure in the past to finally identify anidentical device. The accumulated data is obtained by collecting anaverage of or an increase and decrease trend in a traffic amount withina certain period of time, and the like in types or models of devices,and is assumed to be collected before performing device identificationprocessing, i.e., in advance. In the technology described in NPL 1, fora device having the highest degree of similarity among degrees ofsimilarity obtained from a feature amount of communication informationof an identification target device and accumulated data of devices, whenthe degree of similarity is less than a certain threshold (new typedetermination threshold), the device is determined to be a new typedevice for which there is no data accumulated in advance.

CITATION LIST Non Patent Literature

NPL 1: HIROFUMI NOGUCHI, MISAO KATAOKA, and YOJI YAMATO; “DeviceIdentification Based on Communication Analysis for the Internet ofThings,” IEEE Access, Volume 7, 2019, P. 52903-52912

SUMMARY OF THE INVENTION Technical Problem

In order to apply machine learning to and automate the determinationtechnique of the new type device, which is the technology described inNPL 1 described above, there are two problems to be improved, asdescribed below. Note that the present invention aims to perform moreaccurate classification than that of the related art by applying machinelearning to determination of a new type device.

A first problem is a problem brought to the fore when machine learningis applied to calculation of a degree of similarity between anidentification target and accumulated data. In the technology describedin NPL 1, it is assumed that a degree of similarity is expressed in anabsolute amount, and, for example, when feature amounts of a device A, adevice B, and a device C are previously collected as accumulated data,similarities between the respective devices and an identification targetneed to be calculated as numerical values of 10, 80, and 90,respectively. On the other hand, in classification processing usingsoftware of supervised learning, which is one of the machine learning, arelative amount, in which the sum of degrees of certainty (extents ofcertainty) classified into candidate labels (device A, device B, anddevice C in this example) is 100%, is calculated. For example, values of10%, 40%, and 50% are obtained for the three candidates, respectively.That is, while such values can indicate superiority or inferiority inthe population, it is impossible to determine, as absolute amounts, howsimilar the candidates and the identification target are, by using thevalues. Due to this, with simple application of machine learning, it isimpossible to determine a new type by the new type determinationthreshold.

A second problem is a problem related to presetting of a new typedetermination threshold. An appropriate new type determination thresholdis different depending on how many similar devices are present in anetwork environment. For example, even if an identification target isactually a new type device, when there are a large number of devices ofthe same type in the environment, the degree of similarity with existingdevices is calculated to be high, and thus the new type determinationthreshold has to be set to be high in order to correctly determine a newtype. On the other hand, if the new type determination threshold is toohigh, the same device cannot be correctly determined. This is becausecommunication information such as a packet length has a variation andthus even feature amounts obtained from the same devices do not becomeexactly identical. In a situation where it is not clear what type ofdevices are connected, it is difficult to automatically set anappropriate threshold.

The present invention is made in light of the foregoing, and an objectof the present invention is to enable highly accurate determination of atype or a model of a device connected to a network or whether the deviceis a new type, by machine learning.

Means for Solving the Problem

A device identification apparatus according to the present inventionincludes: a communication information collection unit configured toacquire communication information on existing devices indicating devicesthat exist and are connected to a network in models or types and acquirecommunication information on an identification target device indicatinga device to be identified; a feature amount generation unit configuredto make the communication information of the existing devices featureamounts at a predetermined time interval, and generate feature amountdata by assigning labels for identifying the models or the types to thefeature amounts to use the feature amount data as first training data,make the communication information of the identification target devicefeature amounts at a predetermined time interval, and generate featureamount data by assigning a dummy label to the feature amounts to use thefeature amount data as second training data, and further acquirecommunication information of the identification target device, make thecommunication information feature amounts at a predetermined timeinterval to generate feature amount data, and use the generated featureamount data as identification data; a machine learning unit configuredto cause a learning engine to learn the first training data with thelabels and the second training data with the dummy label, and input theidentification data to the learning engine that has learned the firsttraining data and the second training data to classify the featureamount data of the identification target device into the labelsincluding the dummy label; a degree-of-similarity calculation unitconfigured to calculate, based on the number of pieces of the featureamount data classified into the labels and the number of pieces of thefeature amount data classified into the dummy label, a degree ofsimilarity between a model or a type of the identification target deviceand each of the models or types indicated by the labels; and a deviceidentification unit configured to determine, for the degree ofsimilarity of each of the labels, whether there is a label having adegree of similarity equal to or greater than a predetermined new typedetermination threshold, in accordance with a determination that thereis a label having a degree of similarity equal to or greater than thepredetermined new type determination threshold, identify theidentification target device as an existing device of a model or typeindicated by the label, and in accordance with a determination thatthere is no label having a degree of similarity equal to or greater thanthe predetermined new type determination threshold, identify theidentification target device as a new type device different from themodels or types indicated by the existing labels.

Effects of the Invention

According to the present invention, it is possible to determine a typeor model of a device connected to a network or whether the device is anew type with high accuracy by machine learning.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a deviceidentification apparatus according to a present embodiment.

FIG. 2 is a diagram for explaining processing of classifying featureamount data of an identification target device into labels (including adummy label) by the device identification apparatus (machine learningunit) according to the present embodiment.

FIG. 3 is a diagram for explaining calculation of a degree of similarityfor each label (model) according to the device identification apparatus(degree-of-similarity calculation unit) according to the presentembodiment.

FIG. 4 is a diagram for explaining setting of a new type determinationthreshold (new type determination threshold settable range) by thedevice identification apparatus (new type determination thresholdcalculation unit) according to the present embodiment.

FIG. 5 is a diagram for explaining a problem of the setting of a newtype determination threshold by the device identification apparatus (newtype determination threshold calculation unit) according to the presentembodiment.

FIG. 6 is a diagram for explaining a specific example of calculation ofa new type determination threshold by the device identificationapparatus (new type determination threshold calculation unit) accordingto the present embodiment.

FIG. 7 is a diagram for explaining a specific example of calculation ofa new type determination threshold by the device identificationapparatus (new type determination threshold calculation unit) accordingto the present embodiment.

FIG. 8 is a diagram for explaining a specific example of calculation ofa new type determination threshold by the device identificationapparatus (new type determination threshold calculation unit) accordingto the present embodiment.

FIG. 9 is a flowchart of device identification processing executed bythe device identification apparatus according to the present embodiment.

FIG. 10 is a flowchart (1) of new type determination thresholdcalculation processing executed by the device identification apparatusaccording to the present embodiment.

FIG. 11 is a flowchart (2) of the new type determination thresholdcalculation processing executed by the device identification apparatusaccording to the present embodiment.

FIG. 12 is a hardware configuration diagram illustrating an example of acomputer that implements functions of the device identificationapparatus according to the present embodiment.

DESCRIPTION OF EMBODIMENTS

Next, an embodiment of the present invention (hereinafter referred to as“present embodiment”) will be described.

FIG. 1 is a block diagram illustrating a configuration of a deviceidentification apparatus 1 according to the present embodiment.

The device identification apparatus 1 acquires communication informationof devices that are communicatively connected via a network, and thelike, and determines whether a device to be identified is a new typedevice (new model (or class) that does not exist as models (or classes)of devices that exist).

The device identification apparatus 1 performs <1> calculation of adegree of similarity of a device to be identified by using machinelearning and <2> automatic setting of a new type determination thresholdto implement a determination of whether the device is a new type device.Note that in the present embodiment described below, the descriptionwill be mainly given on the premise that the device identificationapparatus 1 determines a “model” of a device to be identified. However,the device identification apparatus 1 can also determine a “class” ofthe identification target device by a similar technique.

In calculating a degree of similarity of a device, the deviceidentification apparatus 1 assigns a new label (dummy label) differentfrom labels assigned to feature amount data of communication informationof models accumulated until then to feature amount data (details will bedescribed later) of communication information of a device to beidentified, and uses the data with the new label as training data totrain a learning engine. The device identification apparatus 1 thenuses, as a degree of similarity of a device, a value obtained bydividing the number of feature amounts (the number of pieces of featureamount data) classified into each label by the number of feature amountsclassified into the dummy label, the resulting value being expressed inpercentage. In addition, the device identification apparatus 1calculates the degree of similarity under two conditions, that is, whenthere is training data corresponding to the device to be identified andwhen the training data does not exist, and calculates a new typedetermination threshold from the calculated degrees of similarity underthe two conditions (details will be described later).

In this way, the device identification apparatus 1 automaticallycalculates a new type determination threshold for identifying a new typedevice on the basis of communication information of devices. Then, thedevice identification apparatus 1 compares the value of the degree ofsimilarity calculated based on the communication information of thedevice to be identified and the communication information of devicesalready accumulated with the new type determination threshold, so thatit is possible to determine whether the device to be identified is a newtype device.

Next, functions of the device identification apparatus 1 will bespecifically described with reference to FIG. 1 .

The device identification apparatus 1 is realized by a computerincluding a control unit 10, an input/output unit 11, and a storage unit12.

The input/output unit 11 is composed of a communication interface fortransmitting and receiving information, and an input/output interfacefor transmitting and receiving information to and from an inputapparatus such as a touch panel and a keyboard and an output apparatussuch as a monitor.

The storage unit 12 is composed of a flash memory, a hard disk, a randomaccess memory (RAM), and the like. In the storage unit 12 of the deviceidentification apparatus 1, information of feature amounts extractedfrom communication information of devices, to which a label is assignedfor each model (or class), is stored in an accumulated data database(DB) 120.

As illustrated in FIG. 1 , the control unit 10 includes a communicationinformation collection unit 101, a feature amount generation unit 102, amachine learning unit 103, a degree-of-similarity calculation unit 104,a device identification unit 105, and a new type determination thresholdcalculation unit 106.

The communication information collection unit 101 acquires communicationinformation in models of devices. Here, the communication information isinformation obtained by collecting information such as a packet lengthof a packet transmitted and received by each device in the network, adestination port number, or a window size of a header for apredetermined period of time (e.g., 10 minutes). Note that when thecommunication information collection unit 101 acquires communicationinformation in classes of devices, the communication informationcollection unit 101 also acquires information similar to that in models(e.g., packet length, destination port number, window size, and thelike).

The communication information collection unit 101 may acquire thecommunication information directly from devices, or may acquire thecommunication information from a network management apparatus (notillustrated) or the like that manages the devices.

When a device to be identified (hereinafter, sometimes referred to as an“identification target device”) is connected to the network, thecommunication information collection unit 101 collects communicationinformation transmitted and received by the identification target devicefor a predetermined period of time (e.g., 10 minutes). Note that thecommunication information collection unit 101 collects the communicationinformation of the identification target device for collecting trainingdata and for collecting identification data.

Note that the communication information collection unit 101 acquiresinformation that the identification target device is connected to thenetwork, for example, from the network management apparatus or the like.Furthermore, the communication information collection unit 101 mayacquire the communication information of the identification targetdevice in two separate timings for the training data and for theidentification data, and may acquire the communication informationcontinuously for the training data and for the identification data anddivide the acquired information.

The feature amount generation unit 102 acquires the communicationinformation of existing (identified) devices, and performs the followingprocessing.

The feature amount generation unit 102 makes the communicationinformation in models of devices acquired for a predetermined period oftime (e.g., 10 minutes) feature amounts (for example, calculates anaverage value) at a predetermined time interval (a predetermined cycle,e.g., 60 seconds cycle) and assigns labels to the feature amounts togenerate the feature amount data (communication information made featureamounts). Then, the feature amount generation unit 102 stores thefeature amount data with labels as training data (first training data)in the accumulated data DB 120 in the storage unit 12. Here, the labelsare information identifying models (or classes), and for example, ModelA, Model B, and Model C are assigned as the labels (existing labels).

Note that the processing described above by the feature amountgeneration unit 102 is performed in advance before processing thecommunication information of the identification target device.

Furthermore, the feature amount generation unit 102 acquires thecommunication information of the identification target device, andperforms the following processing. When the feature amount generationunit 102 acquires the communication information of the identificationtarget device for a predetermined period of time (for example, 10minutes), the feature amount generation unit 102 makes the communicationinformation feature amounts at a predetermined time interval (forexample, 60 second s cycle) and assigns a dummy label to the featureamounts to and generate the feature amount data. Then, the featureamount generation unit 102 stores the feature amount data with the dummylabel as training data (second training data) in the accumulated data DB120 in the storage unit 12.

Furthermore, when the feature amount generation unit 102 furtheracquires the communication information of the identification targetdevice for a predetermined period of time (for example, 10 minutes), thefeature amount generation unit 102 makes the communication informationfeature amounts at a predetermined time interval (for example, 60seconds cycle) to generate feature amount data. The feature amountgeneration unit 102 stores the feature amount data as identificationdata in the storage unit 12.

The machine learning unit 103 includes a learning engine 3 (machinelearning algorithm). As the machine learning algorithm, a neuralnetwork, a logistic regression, a support vector machine (SVM), or thelike can be used, for example.

The machine learning unit 103 causes the learning engine 3 to learntraining data (feature amount data) with labels (models) of devices. Inaddition, the machine learning unit 103 causes the learning engine 3 tolearn the feature amount data with the dummy label of the identificationtarget device as the training data.

In addition, the machine learning unit 103 inputs feature amount data tobe identified (identification data) to the learning engine 3, thefeature amount data being obtained by making the communicationinformation of the identification target device feature amounts, andclassifies the feature amount data into labels (including the dummylabel).

As a result, as illustrated in FIG. 2 , the feature amount data(identification data) of the identification target device is classifiedinto the label “Model A”, the label “Model B”, the label “Model C”, orthe dummy label.

With reference again to FIG. 1 , the degree-of-similarity calculationunit 104 performs calculation in which based on the number of pieces ofthe feature amount data classified into each label by the machinelearning unit 103, the number of pieces of the feature amount dataclassified into a label (existing label) other than the dummy label isdivided by the number of pieces of the feature amount data classifiedinto the dummy label and the resulting value is expressed in percentageto be used as a degree of similarity of the device.

For example, as illustrated in FIG. 3 , when the number of pieces of thefeature amount data classified into the dummy label is “6”, the numberof pieces of the feature amount data classified into Model A is “1”, thenumber of pieces of the feature amount data classified into Model B is“0”, and the number of pieces of the feature amount data classified intoModel C is “5,” degrees of similarity of the models indicated by therespective labels are calculated as follows.

-   Degree of similarity of label “Model A” . . . 1/6×100=17%-   Degree of similarity of label “Model B” . . . 0/6×100=0%-   Degree of similarity of label “Model C” . . . 5/6×100=83%

As a result, when training data of a model corresponding to theidentification target device is already present, half of the featureamount data is theoretically classified into each of the dummy label anda label (model) of a corresponding existing device, resulting in adegree of similarity of near 100%. On the other hand, when there is notraining data of the model corresponding to the identification targetdevice, almost all the feature amount data (identification data) isclassified into the dummy label, resulting in a very small value even ina degree of similarity of the label (model) corresponding to the mostsimilar device.

With reference again to FIG. 1 , the device identification unit 105 usesa new type determination threshold calculated by the new typedetermination threshold calculation unit 106 described later todetermine whether or not there is a label (model) indicating a degree ofsimilarity equal to or greater than the new type determinationthreshold, among the values of degrees of similarity of the labelscalculated by the degree-of-similarity calculation unit 104. Then, whenthere is a label (model) indicating a degree of similarity equal to orgreater than the new type determination threshold, the deviceidentification unit 105 determines that the identification target deviceis a device of the model (the same model as the model of an existingdevice).

On the other hand, when there is no model indicating a degree ofsimilarity equal to or greater than the new type determinationthreshold, the device identification unit 105 determines that theidentification target device with the dummy label is a new type device(for example, “Model D”).

Note that when the device identification unit 105 determines that theidentification target device is a new type device, the deviceidentification unit 105 outputs this information to the new typedetermination threshold calculation unit 106. As a result, the new typedetermination threshold calculation unit 106 executes the new typedetermination threshold calculation processing again, and updates thenew type determination threshold.

When the device identification unit 105 determines that theidentification target device is a new type device, the new typedetermination threshold calculation unit 106 executes the new typedetermination threshold calculation processing to calculate and updatethe new type determination threshold.

The new type determination threshold is a threshold for the degree ofsimilarity, which is provided for determining whether the identificationtarget device is a new type (new model) for which there is noaccumulated data for learning (training data), or of an existing modelfor which there is already training data. A method for calculating a newtype determination threshold by the new type determination thresholdcalculation unit 106 will be described below.

Note that the communication information collection unit 101 collectscommunication information transmitted and received for a predeterminedperiod of time (e.g., 10 minutes) from any number (number of models,e.g., Model A, Model B, Model C) of devices in advance. Then, thefeature amount generation unit 102 makes the communication informationin models feature amounts (for example, calculates an average value) ata predetermined time interval (predetermined cycle, e.g., 60 secondscycle) to generate feature amount data, assigns labels (e.g., Model A,Model B, Model C) to the feature amount data, and stores the featureamount data with labels in the storage unit 12.

For example, the feature amount generation unit 102 acquires, from thecommunication information collection unit 101, data (communicationinformation) captured for 10 minutes for each device, and makes the data(communication information) a feature amount at 60 seconds cycle togenerate 10 pieces of feature amount data per device (model).

Note that instead of newly generating the feature amount data for eachmodel, the new type determination threshold calculation unit 106 may usethe feature amount data of each model stored in the accumulated data DB120 for the new type determination threshold calculation processing.

In the above-described state, the new type determination thresholdcalculation unit 106 divides the feature amount data of each model intotraining data and test data. The division ratio is arbitrary, but ingeneral machine learning, it is often set to be 80% of training data and20% of test data.

Next, the new type determination threshold calculation unit 106 performslearning by the machine learning unit 103 (learning engine 3) using thetraining data of all the labels (models). Then, degrees of similarityfor devices of the labels (models) are calculated by the same procedureas the degree of similarity calculation processing described above.

Specifically, the new type determination threshold calculation unit 106selects one label (model) and divides the test data of the model intotwo. The division ratio is arbitrary, but the test data is divided, forexample, in half. The new type determination threshold calculation unit106 assigns a dummy label to one of the divided pieces of the test dataand causes the learning engine 3 to learn the test data with the dummylabel. Then, the new type determination threshold calculation unit 106inputs the remaining piece of the test data to the learning engine 3,and classifies the input test data into labels (including the dummylabel).

Subsequently, similarly to the processing of the degree-of-similaritycalculation unit 104, the new type determination threshold calculationunit 106 performs calculation in which based on the number of pieces ofthe feature amount data classified into each label, the number of piecesof the feature amount data classified into a label is divided by thenumber of pieces of the feature amount data classified into the dummylabel and the resulting value is expressed in percentage to be used as adegree of similarity of the device.

Then, the new type determination threshold calculation unit 106 extractsthe degree of similarity of a correct (selected) label (model) as the“degree of similarity of the device when there is training data”. Thenew type determination threshold calculation unit 106 performs thisprocessing while selecting each of all models to extract a degree ofsimilarity of a correct label (model) when there is training data.

Next, the new type determination threshold calculation unit 106 excludestraining data belonging to one label arbitrarily selected among all thetraining data and performs learning by the learning engine 3. Then, thenew type determination threshold calculation unit 106 calculates degreesof similarity for devices of respective models by the same procedure asthe degree of similarity calculation processing described above.

Specifically, for one selected label, the new type determinationthreshold calculation unit 106 divides test data for the label (model)into two. The division ratio is arbitrary, but the test data is divided,for example, in half. The new type determination threshold calculationunit 106 assigns a dummy label to one of the divided pieces of the testdata and causes the learning engine 3 to learn the test data with thedummy label. Then, the new type determination threshold calculation unit106 inputs the remaining piece of the test data to the learning engine3, and classifies the input test data into labels (including the dummylabel).

Subsequently, similarly to the degree-of-similarity calculation unit104, the new type determination threshold calculation unit 106 performscalculation in which based on the number of pieces of the feature amountdata classified into each label, the number of pieces of the featureamount data classified into a label is divided by the number of piecesof the feature amount data classified into the dummy label and theresulting value is expressed in percentage to be used as a degree ofsimilarity of the device.

Then, the new type determination threshold calculation unit 106 extractsa degree of similarity of the most similar label among degrees ofsimilarity of labels when there is no training data. The new typedetermination threshold calculation unit 106 also performs similarprocessing for all other labels, and extracts the “degree of similarityof the most similar device when there is no training data”.

The new type determination threshold calculation unit 106 sets a newtype determination threshold for each model, extracted above process, soas to fall between the “degree of similarity of the device when there istraining data” and the “degree of similarity of the most similar devicewhen there is no training data” (see FIG. 4 ). Note that in FIG. 4 , asymbol “⋄” indicates a degree of similarity with a correct label whenthere is training data (“p” described below). Furthermore, a symbol “Δ”indicates a degree of similarity of the most similar device when thereis no training data (“q” described below). As illustrated in FIG. 4 , inall of Model A, Model B, and Model C, a range that falls between the“degree of similarity of a device when there is training data” (⋄) andthe “degree of similarity of the most similar device when there is notraining data” (Δ) is ideally a new type determination thresholdsettable range.

Specifically, a new type determination threshold is calculated, forexample, so that an intermediate value between the degree of similarityof a device when there is training data and the degree of similarity ofthe most similar device when there is no training data satisfies newtype determination threshold calculation equations of Equations (1) and(2) which will be described below.

This is because it is not possible to ensure that there is a similartendency for degrees of similarity also in an actual device of new typeand thus, when a margin is widely left on either side of new typedetermination and existing type determination, that is, an intermediatevalue is taken, it is possible to absorb these factors.

$\begin{matrix}\left\lbrack {{Math}.1} \right\rbrack &  \\{{F(x)} = {\sum\limits_{i = 1}^{n}\left( \left| {\left( {p_{i} - x} \right) - \left( {x - q_{i}} \right)} \right| \right)}} & {{Equation}(1)}\end{matrix}$ $\begin{matrix}{x = {{{argmin}F}(x)}} & {{Equation}(2)}\end{matrix}$

Where x is a new type determination threshold, p is a degree ofsimilarity with a correct label when there is training data, q is adegree of similarity of the most similar device when there is notraining data, and n is the number of labels. In addition, Equation (2)is meant to calculate x that minimizes F(x).

In addition, calculated degrees of similarity vary from model to modeland thus, it may be impossible to set a new type determination thresholdthat satisfies the conditions for all models (see FIG. 5 ). In otherwords, in FIG. 4 , a common new type determination threshold settablerange can be provided for all the models. On the other hand, in theexample illustrated in FIG. 5 , settable ranges do not overlap for ModelB and Model C and it is thus impossible to simply set a new typedetermination threshold.

In such a case as well, the new type determination threshold calculationunit 106 sets a new type determination threshold by the technique usingEquation (1) and Equation (2) as described above, so that a new typedetermination threshold capable of performing correct identification foras many models as possible can be calculated.

FIGS. 6 to 8 are diagrams for conceptually explaining processing inwhich the new type determination threshold calculation unit 106calculates a new type determination threshold using Equation (1) andEquation (2) described above.

Here, it is assumed that the degree of similarity (p) with a correctlabel when there is training data for Model A is “90 (%)” and the degreeof similarity (q) of the most similar device when there is no trainingdata for Model A is “10 (%)”. In addition, it is assumed that the degreeof similarity (p) with a correct label when there is training data forModel B is “70 (%)” and the degree of similarity (q) of the most similardevice when there is no training data for Model B is “50 (%)”. Inaddition, it is assumed that the degree of similarity (p) with a correctlabel when there is training data for Model C is “30 (%)” and the degreeof similarity (q) of the most similar device when there is no trainingdata for Model C is “15 (%)”.

In a case where the new type determination threshold x=40 holds asillustrated in FIG. 6 , when (|(p_(i)−x)-(x−q_(i))|) of Equation (1) iscalculated for each model, the following values are obtained.

-   Model A: (|(90−40)−(40−10)|)=20-   Model B: (|(70−40)−(40−50)|)=40-   Model C: (|(30−40)−(40−15)|)=35

Thus, F(x)=20+40+35=95 is obtained.

Similarly, FIG. 7 illustrates a case where the new type determinationthreshold x=50 holds, and F(x)=75 is obtained. Furthermore, FIG. 8illustrates a case where the new type determination threshold x=30holds, and F(x)=115 is obtained.

The new type determination threshold calculation unit 106 can determinethe new type determination threshold x that minimizes F(x) by searchingin this manner.

Note that as described above, when the device identification unit 105determines that an identification target device is a new type devicebased on the new type determination threshold, the new typedetermination threshold calculation unit 106 adds the device to existingdevices, performs similar processing (new type determination thresholdcalculation processing), and updates the new type determinationthreshold.

Flow of Processing

Next, a flow of processing executed by the device identificationapparatus 1 will be described.

Device Identification Processing

FIG. 9 is a flowchart illustrating device identification processingexecuted by the device identification apparatus 1 according to thepresent embodiment. The device identification apparatus 1 utilizesmachine learning to calculate a degree of similarity based on a featureamount of a device to be identified and feature amounts of modelscorresponding to existing devices, and compares the calculated degree ofsimilarity with a new type determination threshold to determine whetherthe identification target device is a device of the same model as theexisting models or a new type device. Detailed description will be givenbelow.

First, the communication information collection unit 101 of the deviceidentification apparatus 1 acquires communication information in modelsof devices (step S1). The communication information collection unit 101acquires the communication information of the devices for apredetermined period of time (for example, 10 minutes) in models.

Next, the feature amount generation unit 102 makes the communicationinformation in models of the devices collected by the communicationinformation collection unit 101 feature amounts at a predetermined timeinterval (for example, 60 seconds cycle), and assigns labels of modelsof the devices (e.g., Model A, Model B, Model C) to the feature amountsto generate feature amount data (training data) (step S2). Then, thefeature amount generation unit 102 stores the feature amount data withlabels as training data (first training data) in the accumulated data DB120 in the storage unit 12.

Note that steps S1 and S2 are performed in advance before communicationinformation of the identification target device is acquired.

Next, the communication information collection unit 101 collectscommunication information transmitted and received by the identificationtarget device for a predetermined period of time (e.g., 10 minutes)when, for example, the identification target device is connected to thenetwork (step S3).

Subsequently, the feature amount generation unit 102 makes thecommunication information of the identification target device featureamounts at a predetermined time interval (e.g., 60 seconds cycle), andassigns a dummy label to the feature amounts to generate feature amountdata (training data) (step S4). Then, the feature amount generation unit102 stores the feature amount data with the dummy label as training data(second training data) in the accumulated data DB 120 in the storageunit 12.

Next, the communication information collection unit 101 further collectscommunication information transmitted and received by the identificationtarget device for a predetermined period of time (e.g., 10 minutes)(step S5).

Then, the feature amount generation unit 102 makes the communicationinformation of the identification target device feature amounts at apredetermined time interval (e.g., 60 seconds cycle), and generatesfeature amount data to be identified (identification data) (step S6).The feature amount generation unit 102 stores the generated featureamount data as the identification data in the storage unit 12.

Subsequently, the machine learning unit 103 inputs the training datawhich is the feature amount data with the labels of the existing models(Model A, Model B, Model C) and with the dummy label generated by thefeature amount generation unit 102 to the learning engine 3 and causesthe learning engine 3 to learn the training data (step S7). That is, byinputting the feature amount data and the labels assigned thereto asinput data to the learning engine 3, learning is performed in which aparameter of the learning engine 3 is adjusted so that when featureamount data is input, the feature amount data is classified into acorrect label.

Then, the machine learning unit 103 inputs the feature amount data to beidentified (identification data), which is obtained by making thecommunication information of the identification target device featureamounts, to the learning engine 3, and classifies the identificationdata into the labels (including the dummy label) (step S8).

Next, the degree-of-similarity calculation unit 104 calculates degreesof similarity between the model (label) of the identification targetdevice and the models (labels) of the devices based on the number ofpieces of the feature amount data classified into the labels (includingthe dummy label) (step S9).

Specifically, the degree-of-similarity calculation unit 104 performscalculation in which the number of pieces of the feature amount dataclassified into each label (existing label) other than the dummy labelis divided by the number of pieces of the feature amount data classifiedinto the dummy label and the resulting value is expressed in percentageto be used as a degree of similarity of the device (model).

Subsequently, the device identification unit 105 uses a new typedetermination threshold calculated by the new type determinationthreshold calculation unit 106 performing new type determinationthreshold calculation processing illustrated in FIG. 10 and FIG. 11described later to determine whether or not the identification targetdevice is a new type device. Specifically, the device identificationunit 105 determines whether or not there is a label (model) indicating adegree of similarity equal to or greater than the new type determinationthreshold, among values indicating the degrees of similarity of thelabels calculated by the degree-of-similarity calculation unit 104 (stepS10).

Then, in accordance with a determination that there is a label (model)indicating a degree of similarity equal to or greater than the new typedetermination threshold (step S10→Yes), the device identification unit105 determines that the model indicated by the label is the model of theidentification target device. That is, the device identification unit105 determines that the identification target device is of the samemodel as the existing device (step S11).

On the other hand, in accordance with a determination that there is nolabel (model) indicating a degree of similarity equal to or greater thanthe new type determination threshold (step S10→No), the deviceidentification unit 105 determines the identification target device withthe dummy label as a new type device (step S12).

Then, the device identification unit 105 outputs the information, to thenew type determination threshold calculation unit 106, that theidentification target device is determined to be a new type device (stepS13) and ends the processing.

As a result, the new type determination threshold calculation unit 106adds the identification target device determined to be a new type deviceto the existing devices as, for example, Model D, and executes the newtype determination threshold calculation processing again to update thenew type determination threshold.

In this way, the device identification apparatus 1 uses the featureamount data generated from communication information of devices anddetermines labels by machine learning, so that it is possible toclassify a corresponding label of the feature amount data with higheraccuracy. Furthermore, the device identification apparatus 1 candetermine, for an identification target device connected to a network,whether the device is of an existing model (or type) or of a new type,using a degree of similarity that is an absolute amount calculated basedon the number of pieces of the feature amount data classified by thelearning engine 3 rather than a relative amount between labels intowhich devices are classified.

New Type Determination Threshold Calculation Processing

Next, new type determination threshold calculation processing by the newtype determination threshold calculation unit 106 of the deviceidentification apparatus 1 will be described. FIG. 10 and FIG. 11 areflowcharts each illustrating the new type determination thresholdcalculation processing executed by the device identification apparatus 1according to the present embodiment.

Note that the new type determination threshold calculation processing bythe new type determination threshold calculation unit 106 is startedwhen information is acquired from the device identification unit 105that the identification target device is determined to be a new typedevice at step S13 in FIG. 9 .

First, the new type determination threshold calculation unit 106 of thedevice identification apparatus 1 acquires communication information inmodels of devices through the communication information collection unit101 (step S20). The new type determination threshold calculation unit106 acquires the communication information in models of the devices fora predetermined period of time (for example, 10 minutes). Here, in thedevice identification processing in FIG. 9 , the communicationinformation of the devices is acquired in a state where theidentification target device determined to be a new type device is addedas, for example, Model D to existing devices.

Next, the new type determination threshold calculation unit 106 makesthe collected communication information in models of the devices featureamounts through the feature amount generation unit 102 at apredetermined time interval (for example, 60 seconds cycle), and assignslabels of models of the devices (e.g., Model A, Model B, Model C, ModelD) to the feature amounts to generate feature amount data (step S21).Then, the new type determination threshold calculation unit 106 storesthe feature amount data with labels in the storage unit 12 through thefeature amount generation unit 102.

Next, the new type determination threshold calculation unit 106 dividesthe feature amount data of the models into training data and test data(step S22). For example, the new type determination thresholdcalculation unit 106 divides the feature amount data into 80% oftraining data and 20% of test data.

Next, the new type determination threshold calculation unit 106 selectsone label (model) (step S23). Note that the label selected here isreferred to as “label i”. Then, the new type determination thresholdcalculation unit 106 uses the test data of the label i to calculate adegree of similarity between the label i and another label (model) inaccordance with the same procedure as the degree of similaritycalculation processing described above.

Specifically, first, the new type determination threshold calculationunit 106 inputs the training data with labels for all the models (ModelA, Model B, Model C, Model D) to the learning engine 3 and causes thelearning engine 3 to learn the training data (step S24).

Next, the new type determination threshold calculation unit 106 dividesthe test data for the selected label i into two, assigns a dummy labelto one of the divided pieces of the test data (feature amount data),inputs the test data with the dummy label to the learning engine 3, andcauses the learning engine 3 to learn the test data with the dummy label(step S25).

Next, the new type determination threshold calculation unit 106 inputsthe other of the divided pieces of the test data (feature amount data)to the learning engine 3, and classifies the test data into the labels(including the dummy label) (step S26).

Then, the new type determination threshold calculation unit 106 performscalculation in which the number of pieces of the feature amount dataclassified into each label is divided by the number of the featureamount data classified into the dummy label and the resulting value isexpressed in percentage to be used as a degree of similarity of adevice. Then, the new type determination threshold calculation unit 106extracts the degree of similarity of the selected label i as the “degreeof similarity of the device when there is training data” (step S27).

Next, with reference to FIG. 11 , the new type determination thresholdcalculation unit 106 excludes the training data for the selected label iamong all the training data, inputs the training data for the remaininglabels to the learning engine 3, and causes the learning engine 3 tolearn the training data for the remaining labels (step S28).

Then, the new type determination threshold calculation unit 106 dividesthe test data for the selected label i into two, assigns a dummy labelto one of the divided pieces of the test data (feature amount data),inputs the test data with the dummy label to the learning engine 3, andcauses the learning engine 3 to learn the test data with the dummy label(step S29).

Subsequently, the new type determination threshold calculation unit 106inputs the other of the divided pieces of the test data (feature amountdata) to the learning engine 3, and classifies the test data into thelabels (including the dummy label) (step S30).

Then, the new type determination threshold calculation unit 106 performscalculation in which the number of pieces of the feature amount dataclassified into each label is divided by the number of the featureamount data classified into the dummy label and the resulting value isexpressed in percentage to be used as a degree of similarity of adevice. Then, the new type determination threshold calculation unit 106extracts the degree of similarity of the most similar label amongdegrees of similarity of the labels as the “degree of similarity of themost similar device when there is no training data” (step S31).

Subsequently, the new type determination threshold calculation unit 106determines whether or not there is a label that has not yet beenselected (step S32). In accordance with a determination that there is alabel that has not yet been selected (step S32→Yes), the processingreturns to step S23 in FIG. 10 and continues.

On the other hand, in accordance with a determination that all thelabels have been selected (step S3→No), the processing proceeds to thenext step S33.

In step S33, the new type determination threshold calculation unit 106uses the “degree of similarity of the device when there is trainingdata” for each extracted model and the “degree of similarity of the mostsimilar device when there is no training data” to calculate a new typedetermination threshold that satisfies the new type determinationthreshold calculation equations of Equation (1) and Equation (2)described above, and ends the processing.

In this way, the device identification apparatus 1 according to thepresent embodiment automatically calculates a new type determinationthreshold for identifying a new type device based on feature amounts ofcommunication information of devices. The device identificationapparatus 1 compares the value of the degree of similarity calculatedbased on the feature amount of a device to be identified and the featureamounts of devices that have been accumulated in advance with the newtype determination threshold, so that it is possible to determinewhether or not the device to be identified is a new type device.

Hardware Configuration

The device identification apparatus 1 according to the presentembodiment is realized by a computer 900 configured as illustrated inFIG. 12 , for example.

FIG. 12 is a hardware configuration diagram illustrating an example ofthe computer 900 that implements functions of the device identificationapparatus 1 according to the present embodiment. The computer 900includes a central processing unit (CPU) 901, a read only memory (ROM)902, a random access memory (RAM) 903, a hard disk drive (HDD) 904, aninput-output interface (I/F) 905, a communication I/F 906, and a mediaI/F 907.

The CPU 901 operates in accordance with a program stored in the ROM 902or the HDD 904, and performs control with the control unit 10 of FIG. 1. The ROM 902 stores a boot program that is executed by the CPU 901 whenthe computer 900 is activated, a program for the hardware of thecomputer 900 and the like.

The CPU 901 controls an input apparatus 910 such as a mouse and akeyboard, and an output apparatus 911 such as a display and a printerthrough the input-output I/F 905. Through the input-output I/F 905, theCPU 901 acquires data from the input apparatus 910, and outputs thegenerated data to the output apparatus 911.

The HDD 904 stores a program (device identification program) executed bythe CPU 901, data used by the program, and the like. The communicationI/F 906 receives data from another apparatus (not illustrated) (such asa network management apparatus) through a communication network (such asthe network 5) and outputs it to the CPU 901, and transmits datagenerated by the CPU 901 to another apparatus through the communicationnetwork.

The media I/F 907 reads a program (device identification program) ordata stored in a recording medium 912, and outputs it to the CPU 901through the RAM 903. The CPU 901 loads, in the RAM 903, a program for anintended process from the recording medium 912 through the media I/F907, and executes the loaded program. The recording medium 912 is anoptical recording medium such as a digital versatile disc (DVD) and aphase change rewritable disk (PD), a magneto-optical recording mediumsuch as a magneto optical disk (MO), a magnetic recording medium, aconductor memory tape medium, a semiconductor memory or the like.

For example, when the computer 900 functions as the deviceidentification apparatus 1 according to the embodiment, the CPU 901 ofthe computer 900 executes a program loaded on the RAM 903 to implementthe function of the device identification apparatus 1. In addition, theHDD 904 stores data in the RAM 903. The CPU 901 reads a program for anintended process from the recording medium 912 and executes it.Furthermore, the CPU 901 may read a program for an intended process fromanother apparatus through the communication network (the network 5).

EFFECTS OF THE INVENTION

Effects of the device identification apparatus are described below.

The device identification apparatus 1 according to the present inventionincludes: a communication information collection unit 101 configured toacquire communication information on existing devices indicating devicesthat exist and are connected to a network in models or types and acquirecommunication information on an identification target device indicatinga device to be identified; a feature amount generation unit 102configured to make the communication information of the existing devicesfeature amounts at a predetermined time interval, and generate featureamount data by assigning labels for identifying the models or types tothe feature amounts to use the feature amount data as first trainingdata, make the communication information of the identification targetdevice feature amounts at a predetermined time interval, and generatefeature amount data by assigning a dummy label to the feature amounts,to use the feature amount data as second training data, and furtheracquire communication information of the identification target device,make the communication information feature amounts at a predeterminedtime interval to generate feature amount data, and use the generatedfeature amount data as identification data; a machine learning unit 103configured to cause a learning engine 3 to learn the first training datawith the labels and the second training data with the dummy label, andinput the identification data to the learning engine 3 that has learnedthe first training data and the second training data 1 to classify thefeature amount data of the identification target device into the labelsincluding the dummy label; a degree-of-similarity calculation unit 104configured to calculate, based on the number of pieces of the featureamount data classified into the labels and the number of pieces of thefeature amount data classified into the dummy label, a degree ofsimilarity between a model or a type of the identification target deviceand each of the models or types indicated by the labels; and a deviceidentification unit 105 configured to determine, for the degree ofsimilarity of each of the labels, whether there is a label having adegree of similarity equal to or greater than a predetermined new typedetermination threshold, in accordance with a determination that thereis a label having a degree of similarity equal to or greater than thepredetermined new type determination threshold, identify theidentification target device as an existing device of a model or typeindicated by the label, and in accordance with a determination thatthere is no label having a degree of similarity equal to or greater thanthe predetermined new type determination threshold, identify theidentification target device as a new type device different from themodels or types indicated by the existing labels.

With this configuration, the device identification apparatus 1 candetermine whether the device to be identified connected to the networkis an existing model or type of device or a new type device by machinelearning with high accuracy.

The device identification apparatus 1 further includes a new typedetermination threshold calculation unit 106 configured to calculate thepredetermined new type determination threshold, wherein the new typedetermination threshold calculation unit 106 makes the communicationinformation of the existing devices at a predetermined time intervalfeature amounts, assigns the labels to the feature amounts to generatefeature amount data, divides each piece of the generated feature amountdata with the labels into training data and test data, performsprocessing of selecting any one of the labels, further dividing the testdata with the selected label into two, assigning a dummy label to one ofthe test data divided into two, causing the learning engine to learn thetest data with the dummy label and the training data divided for each ofthe labels, and inputting the other of the test data divided into two tothe learning engine that has learned the test data with the dummy labeland the training data divided for each of the labels to classify thetest data with the selected label into the labels including the dummylabel, calculating a degree of similarity of the selected label toextract the degree of similarity as a degree of similarity of the devicewhen there is training data, for each of the labels, performs processingof selecting any one of the labels, further dividing the test data ofthe selected label into two, assigning a dummy label to one of the testdata divided into two, causing the learning engine to learn the testdata with the dummy label and training data except training data of theselected label among the training data divided for the labels, inputtingthe other of the test data divided into two to the learning engine thathas learned the test data with the dummy label and the training dataexcept the training data of the selected label among the training datadivided for the labels to classify the test data of the selected labelinto the labels including the dummy label and excluding the selectedlabel, and calculating degrees of similarity for the labels to extract alabel having the highest value among the calculated degrees ofsimilarity as a degree of similarity of a device most similar when thereis no training data, for each of the label, and calculates, for each ofthe labels, the new type determination threshold to be an intermediatevalue between a degree of similarity of a device when the training dataexists and a degree of similarity of the most similar device when thetraining data does not exist.

In this way, the device identification apparatus 1 can set a moreappropriate new type determination threshold based on the communicationinformation of the existing devices on the network so as to be anintermediate value between the degree of similarity of the device whenthere is training data and the degree of similarity of the most similardevice when there is no training data, for each label.

In the device identification apparatus 1, the device identification unit105, upon identifying the identification target device as a new typedevice, outputs information on the device of new type to the new typedetermination threshold calculation unit 106, and the new typedetermination threshold calculation unit 106 adds the communicationinformation of the device of new type to the communication informationof the existing devices to perform the new type determination thresholdcalculating processing and updates the new type determination threshold.

In this way, when the identification target device is identified as anew type device, the device identification apparatus 1 can perform thenew type determination threshold calculation processing by adding thedevice of new type to existing devices. Thus, the device identificationapparatus 1 can automatically set an appropriate new type determinationthreshold.

REFERENCE SIGNS LIST

-   1 Device identification apparatus-   3 Learning Engine-   10 Control unit-   11 Input/output unit-   12 Storage unit-   101 Communication information collection unit-   102 Feature amount generation unit-   103 Machine learning unit-   104 Degree-of-similarity calculation unit-   105 Device identification unit-   106 New type determination threshold calculation unit-   120 Accumulated data DB

The invention claimed is:
 1. A device identification apparatuscomprising: a communication information collection unit, including oneor more processors, configured to acquire communication information onexisting devices indicating devices that exist and are connected to anetwork in models or types and acquire communication information on anidentification target device indicating a device to be identified; afeature amount generation unit, including one or more processors,configured to make the communication information of the existing devicesfeature amounts at a predetermined time interval, and generate featureamount data by assigning labels for identifying the models or the typesto the feature amounts to use the feature amount data as first trainingdata, make the communication information of the identification targetdevice feature amounts at a predetermined time interval, and generatefeature amount data by assigning a dummy label to the feature amounts touse the feature amount data as second training data, and further acquirecommunication information of the identification target device, make thecommunication information feature amounts at a predetermined timeinterval to generate feature amount data, and use the generated featureamount data as identification data; a machine learning unit, includingone or more processors, configured to cause a learning engine to learnthe first training data with the labels and the second training datawith the dummy label, and input the identification data to the learningengine that has learned the first training data and the second trainingdata to classify the feature amount data of the identification targetdevice into the labels including the dummy label; a degree-of-similaritycalculation unit, including one or more processors, configured tocalculate, based on the number of pieces of the feature amount dataclassified into the labels and the number of pieces of the featureamount data classified into the dummy label, a degree of similaritybetween a model or a type of the identification target device and eachof the models or types indicated by the labels; and a deviceidentification unit, including one or more processors, configured todetermine, for the degree of similarity of each of the labels, whetherthere is a label having a degree of similarity equal to or greater thana predetermined new type determination threshold, in accordance with adetermination that there is a label having a degree of similarity equalto or greater than the predetermined new type determination threshold,identify the identification target device as an existing device of amodel or type indicated by the label, and in accordance with adetermination that there is no label having a degree of similarity equalto or greater than the predetermined new type determination threshold,identify the identification target device as a new type device differentfrom the models or types indicated by the existing labels.
 2. The deviceidentification apparatus according to claim 1, further comprising a newtype determination threshold calculation unit, including one or moreprocessors, configured to calculate the predetermined new typedetermination threshold, wherein the new type determination thresholdcalculation unit is configured to: make the communication information ofthe existing devices at a predetermined time interval feature amounts,assigns the labels to the feature amounts to generate feature amountdata, divides each piece of the generated feature amount data with thelabels into training data and test data, perform processing of selectingany one of the labels, further dividing the test data with the selectedlabel into two, assigning a dummy label to one of the test data dividedinto two, causing the learning engine to learn the test data with thedummy label and the training data divided for each of the labels,inputting the other of the test data divided into two to the learningengine that has learned the test data with the dummy label and thetraining data divided for each of the labels to classify the test datawith the selected label into the labels including the dummy label,calculating a degree of similarity of the selected label to extract thedegree of similarity as a degree of similarity of the device when thereis training data, for each of the labels, perform processing ofselecting any one of the labels, further dividing the test data of theselected label into two, assigning a dummy label to one of the test datadivided into two, causing the learning engine to learn the test datawith the dummy label and training data except training data of theselected label among the training data divided for the labels, inputtingthe other of the test data divided into two to the learning engine thathas learned the test data with the dummy label and the training dataexcept the training data of the selected label among the training datadivided for the labels to classify the test data of the selected labelinto the labels including the dummy label and excluding the selectedlabel, and calculating degrees of similarity for the labels to extract alabel having the highest value among the calculated degrees ofsimilarity as a degree of similarity of a device most similar when thereis no training data, for each of the label, and calculate, for each ofthe labels, the new type determination threshold to be an intermediatevalue between a degree of similarity of a device when the training dataexists and a degree of similarity of the most similar device when thetraining data does not exist.
 3. The device identification apparatusaccording to claim 2, wherein the device identification unit, uponidentifying the identification target device as the device of new type,is configured to output information on the device of new type to the newtype determination threshold calculation unit, and the new typedetermination threshold calculation unit is configured to add thecommunication information of the device of new type to the communicationinformation of the existing devices to perform calculating processing ofthe new type determination threshold and updates the new typedetermination threshold.
 4. A device identification method by a deviceidentification apparatus that identifies a device connected to anetwork, the method including, at the device identification apparatus:acquiring communication information on existing devices indicatingexisting devices connected to the network in models or types, andacquiring communication information on an identification target deviceindicating a device to be identified; making the communicationinformation of the existing devices feature amounts at a predeterminedtime interval, generating feature amount data by assigning labels foridentifying the models or types to the feature amounts to use thefeature amount data as first training data, making the communicationinformation of the identification target device feature amounts at apredetermined time interval, generating feature amount data by assigninga dummy label to the feature amounts to use the feature amount data assecond training data, and further acquiring communication information ofthe identification target device, and making the communicationinformation feature amounts at a predetermined time interval to generatefeature amount data, and using the generated feature amount data asidentification data; causing a learning engine to learn the firsttraining data with the labels and the second training data with thedummy label, and inputting the identification data to the learningengine that has learned the first training data and the second trainingdata to classify the feature amount data of the identification targetdevice into the labels including the dummy label; calculating, based onthe number of pieces of the feature amount data classified into thelabels and the number of pieces of the feature amount data classifiedinto the dummy label, a degree of similarity between a model or type ofthe identification target device and a model or a type of indicated byeach of the labels; and determining, for the degree of similarity ofeach of the labels, whether there is a label having a degree ofsimilarity equal to or greater than a predetermined new typedetermination threshold, in accordance with a determination that thereis a label having a degree of similarity equal to or greater than thepredetermined new type determination threshold, identifying theidentification target device as an existing device of a model or typeindicated by the label, and in accordance with a determination thatthere is no label having a degree of similarity equal to or greater thanthe predetermined new type determination threshold, identifying theidentification target device as a new type device different from themodels or types indicated by the labels.
 5. The device identificationmethod according to claim 4, wherein the method further includes, at thedevice identification apparatus, making the communication information ofthe existing devices feature amounts at a predetermined time interval,assigning the labels to the feature amounts to generate feature amountdata, dividing each piece of the generated feature amount data with thelabels into training data and test data, performing processing ofselecting any one of the labels, further dividing the test data with theselected label into two, assigning a dummy label to one of the test datadivided into two, causing the learning engine to learn the test datawith the dummy label and the training data divided for each of thelabels, inputting the other of the test data divided into two to thelearning engine that has learned the test data with the dummy label andthe training data divided for each of the labels to classify the testdata with the selected label into the labels including the dummy label,calculating a degree of similarity of the selected label to extract thedegree of similarity as a degree of similarity of the device when thereis training data, for each of the labels, performing processing ofselecting any one of the labels, further dividing the test data of theselected label into two, assigning a dummy label to one of the test datadivided into two, causing the learning engine to learn the test datawith the dummy label and training data except training data of theselected label among the training data divided for the labels, inputtingthe other of the test data divided into two to the learning engine thathas learned the test data with the dummy label and the training dataexcept the training data of the selected label among the training datadivided for the labels to classify test data of the selected label intothe labels including the dummy label and excluding the selected label,and calculating degrees of similarity for the labels to extract a labelhaving the highest value among the calculated degrees of similarity as adegree of similarity of a device most similar when there is no trainingdata, for each of the label, and calculating, for each of the labels,the new type determination threshold to be an intermediate value betweena degree of similarity of a device when the training data exists and adegree of similarity of the most similar device when the training datadoes not exist.
 6. The device identification method according to claim5, including, at the device identification apparatus, upon identifyingthe identification target device as the device of new type, adding thecommunication information of the device of new type to the communicationinformation of the existing devices to perform calculating processing ofthe new type determination threshold and updating the new typedetermination threshold.
 7. A non-transitory computer readable mediumstoring one or more instructions for causing a computer to execute:acquiring communication information on existing devices indicatingexisting devices connected to a network in models or types, andacquiring communication information on an identification target deviceindicating a device to be identified; making the communicationinformation of the existing devices feature amounts at a predeterminedtime interval, generating feature amount data by assigning labels foridentifying the models or types to the feature amounts to use thefeature amount data as first training data, making the communicationinformation of the identification target device feature amounts at apredetermined time interval, generating feature amount data by assigninga dummy label to the feature amounts to use the feature amount data assecond training data, and further acquiring communication information ofthe identification target device, and making the communicationinformation feature amounts at a predetermined time interval to generatefeature amount data, and using the generated feature amount data asidentification data; causing a learning engine to learn the firsttraining data with the labels and the second training data with thedummy label, and inputting the identification data to the learningengine that has learned the first training data and the second trainingdata to classify the feature amount data of the identification targetdevice into the labels including the dummy label; calculating, based onthe number of pieces of the feature amount data classified into thelabels and the number of pieces of the feature amount data classifiedinto the dummy label, a degree of similarity between a model or type ofthe identification target device and a model or a type of indicated byeach of the labels; and determining, for the degree of similarity ofeach of the labels, whether there is a label having a degree ofsimilarity equal to or greater than a predetermined new typedetermination threshold, in accordance with a determination that thereis a label having a degree of similarity equal to or greater than thepredetermined new type determination threshold, identifying theidentification target device as an existing device of a model or typeindicated by the label, and in accordance with a determination thatthere is no label having a degree of similarity equal to or greater thanthe predetermined new type determination threshold, identifying theidentification target device as a new type device different from themodels or types indicated by the labels.
 8. The non-transitory computerreadable medium according to claim 7, wherein the one or moreinstructions cause the computer to execute: making the communicationinformation of the existing devices feature amounts at a predeterminedtime interval, assigning the labels to the feature amounts to generatefeature amount data, dividing each piece of the generated feature amountdata with the labels into training data and test data, performingprocessing of selecting any one of the labels, further dividing the testdata with the selected label into two, assigning a dummy label to one ofthe test data divided into two, causing the learning engine to learn thetest data with the dummy label and the training data divided for each ofthe labels, inputting the other of the test data divided into two to thelearning engine that has learned the test data with the dummy label andthe training data divided for each of the labels to classify the testdata with the selected label into the labels including the dummy label,calculating a degree of similarity of the selected label to extract thedegree of similarity as a degree of similarity of a device when there istraining data, for each of the labels, performing processing ofselecting any one of the labels, further dividing the test data of theselected label into two, assigning a dummy label to one of the test datadivided into two, causing the learning engine to learn the test datawith the dummy label and training data except training data of theselected label among the training data divided for the labels, inputtingthe other of the test data divided into two to the learning engine thathas learned the test data with the dummy label and the training dataexcept the training data of the selected label among the training datadivided for the labels to classify test data of the selected label intothe labels including the dummy label and excluding the selected label,and calculating degrees of similarity for the labels to extract a labelhaving the highest value among the calculated degrees of similarity as adegree of similarity of a device most similar when there is no trainingdata, for each of the label, and calculating, for each of the labels,the new type determination threshold to be an intermediate value betweena degree of similarity of a device when the training data exists and adegree of similarity of the most similar device when the training datadoes not exist.
 9. The non-transitory computer readable medium accordingto claim 8, wherein the one or more instructions cause the computer toexecute: upon identifying the identification target device as the deviceof new type, adding the communication information of the device of newtype to the communication information of the existing devices to performcalculating processing of the new type determination threshold andupdating the new type determination threshold.